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ABSTRACT . - ... „ 

The purpose of this study was to explore whether 
if so how, ongoing measurement and evaluation procedures affect 
instruction that special educators provide and the educational 
decisions that they make. Eighteen experimental ^roup teachers 
employed specific continuous evaluation procedures while 21 cont 
grouj teachers, employed conventional special education measureme 
and evaluation' procedures. During the 18-week ^treatment, the 
teachers' structure of instruction was measured three times, and 
teachers' educational decisions and their assessment procedures- 
surveyed at the middle and end of the treatment. Additionally, a 
termination of the study, teachers' descriptions .and ratings of 
experimental" procedures were assessed. Analyses revealed that us 
systematic, ongoing measurement and evaluation procedures increa, 
the degree of structure in the instruction teachers provide, and 
resulted in more specific and realistic judgments about student 
programs and progress and more responsive pedagogical decisions. 
Experimental teachers alsfc reported that, although time-consumin 
their procedures were feasible and useful. Structure of Instruct 
rating scale, teacher questionnaires, a goal and objective form, 
experimental and contrast training schedule forms are appended. 
(Author/PM) . 
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— —Abstract 

The purpose of this study was to explore whether, and if so how,' 
on-gbing, measurement and evaluation procedures affect the instruction 

0 I 

that special educators provide and the educational decisions that they 
make. Eighteen experimental group teachers employed specific 
continuous evaluation procedures while 21 contrast group teachers 
employed conventional special .education measurement and evaluation 
procedures. During the 18-weok treatment, the teachers' structure of 
instruction was measured three times, and the teachers' educational 
decisions and th6ir assessment procedures were surveyed at the middle 
-and end of the treatment. Additionally, at fhe termination of the 
study, teachers' descriptions and ratings of the experimental 
procedures were assessed. Analyses revealed that using systematic, 
on-going measurement and evaluation procedures (a) - increased the 
degree of e structure, in the instruction teachers provide, and (b) 
resulted in *more specific and realistic judgments about student 
.programs and progress and more responsive jsedagoofcal decisions. 
Experimental teachers also reported that, although time-consuming, 
their procedures were feasible and useful. Implications for special 
education programming are discussed ♦ 



Direct and Frequent Measurement and Evaluation: 
Effects on Instruction and Estimates of-Student Proqress 

'PL 94-142 requires special education teachers to formulate short- 

4 ' 

term objectives and annCial 'goals for students to ensure appropriate 
education and facilitate handicapped pupils* movement toward less 
'restrictive educational settings. Federal law also directs special 

ft 

educators to assess student progress toward specified objectives and 

goals. Despite this apparent concern for and emphasis on student 

— , ^_ ~ < 

evaluation, 94 r 142 does not specify how student progress should be 
assessed; special education practitioners ar§ free to choose whatever 
assessment procedures they Wish," 

Research indicates that special educators tend to employ informal 
observation when toaking decisions about students 1 performance on 
objectives (Potter i Mirkin, 1982). Unfortunately, informal 
observation often* leads to erroneous judgments about levels of 

« 

academic performance and in^Ccur^te conclusions concerning whether 
objectives have been met (Fuchs, Fuchs, 4 Warren, 4982), - 

As an alternative to informal assessment methods, repeated 
curriculum-based measurement and continuous evaluation procedures 
render more objective, accurate data (Fuchs, Mirkin, Denp, Marston, & 
Tindal, 1982). Additionally, such -systematic, on-going measurement 
and evaluation appears to affect positively student 'academic 
achievement -(Beck, 1981; Fuchs, Deno, Mirkin, 1982). Concurrent 
with a better data base and improved pupil gains, one would expect 

*, f ' 

i , 

better teaching and instructional decision making. The purpose of the 
present study was to explore whether more systematic, on-going' 
measurement and K evaluation, in fact, does affect special educators' 

6 



teaching decisions and the instruction they.*provide. Specifically, 
the study examined the following questions: 

• Does repealed, direct measurement and continuous 
evaluation affect the structure of instruction provided* 

v , by special educators? 

• What is the relation between such measurement apd % . 
evaluation and teacners' decisions' about student progress, ' 
goa]s,'and current performance levels? 

• Is there a relation between measurement atfk evaluation 
procedures and' the assessment methods on which teachers 
rely to formulate educational decisions? 

• How do teachers react to using systematic, on-going 
measurement and evaluation procedures? 

Method 

Subjects , , # 

Subjects were 39 New York City public school teachers who 
volunteered to* participate in the project. From their caseloads, 
teachers each selected three or four students for inclusion in the 
study; then, within schools, each teacher was .assigned randomly to 
.either an experimental or a contrast group. Four teachers in 'each 
group had participated in a pilot program during the previous academic 
year. ■ • . , 

In the* experimental group, teachers (3 male, 15 female) had 
taught special education for an average 3.79 years (SD = 2.85).' 
Students in the experimental group (51 male, 13 female) read an 
average '3.48 years below grade level (SD = 1.87). 1 Their age 
appropriate grade averaged 5.79 CSD = 1.66); 49>£ were placed in 
programs for emotionally handicapped students, 32)f in •programs for 
brain injured students, and 19% in resource programs. 

Contrast group 'teachers (2 male, 19 female) had taught 



handicapped children for an average 3*59 years '(StJ' = 2.72). Students 
in the contrast group (57 male, 20 female) read an average 2.35 years 
below grade level (SD = 1..94). Theip age appropriate grade level 
averaged 5.45 (SD = 1.65); 51% were placed in programs for emotionally 
handicapped students, 30% in resource programs, -and 15% equally 
distributed .across programs for physicajly handicapped, brain injured,, 
and educable mentally retarded children'. * * 
Measures . 

Structure of instruction rating scale (SIRS) . The SIRS*(Deno, 
King, Skiba, Sevcik, & Wesson, 1982; see Appendix A) was employed to 
measure the structure of ai\ instructional lesson. The scale is 
completed by an observer following a 20-minute observation of teacher 
interaction with one student. Twelve variables t (Instructional 
Grouping, Teacher-directed , Learning, Active Academic Responding, 
Demonstrating/Prompting, Controlled ^ractice^, Frequency of Correct 
Answers, Independent Practice, Corrections, Positive Consequences, 
Pacingj^^Ofal Reading Practice on Outcome Behavior, and Silent Heading) 
constitute the scale. These variables were selected because research 
supports the relation between each variable and student academic 
achievement (Reith, Polsgrove, & Semmel, 1981; Stevens & Rosen^hine, 
1981). However, the Independent Practice item was dropped during'" 
.analyses betause it, was scored as non-applicable in all but a few 
instances'. Internal - consistency reliabilities (Cronbach's alpha) 
obtained fop the ll-'item scale were .86, .88, and .89 at three 
different measurements. A factor .analysis (Deno et*,al., 1982) 
revealed- that nine items (a-1 but Positive Consequences and Silent 

: : _a : 
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Reading Practice)* Significantly contributed • to one factor called 

i i . * 

. Structure- * , ; • . 

j * Semi-annual teacher questionnaire . A six-item teacher 
questionnaire was designed fey the study (see Appendix. B). On this, 
questionnaire,, teachers (a) described student progress, goals, and 
level'of functioning in reading during the academic .year, and (b) 
checked the assessment, procedures by which they made their judgments 
concerning student progress.' 

End-of-the-year teacher questionnaire . 3 A 12-item teacher" 
questionnaire also was developed (see Appendix 8). This questionnaire 
asked teachers to rate and describe* how .the experimental procedures . 
were different from theiir normal evaluation procedures a^ how th'e 
study procedures were useful. Then," the questionnaire required 
teachers to indicate whether, and if so hoy, they would use the 
experimental pVocedures during the subsequent academic year. 
Procedure * • 

< 

Treatments . ~ 'The experimental treatment is described in 
'Procedures' to Develop fe>d Monitor Progress on IEP Goals (Mirkin, Deno, 
Fuchs, Wesson, 'Tindal, Marston, & Kuehnle, 1981). Employing these" 
* procedures in the .area of reading, the' experimental* group teachers 
first wrote' curriculum-based IEP goals and objectives. Tlrib. -annual 
goal specified the segment of the curriculum and the date on which a. 
student would read 'with proficiency; that is, at a ctrtai^rate and 

* « 

accuracy. .The objective contained supplementary infprmation; it 
indicated the weekly »-;te at which" the student would have to improve 
in order to meet the annual goal (see Goal and Objective Form in 

ERJC .. 9 . 



Appendix C). After setting goals and objectives, teachers developed 

curriculum-based measurement systems, to match specific goals and 

'objectives. Then, they were to measure student's at le^st twice 'weekly 

and # utilize those data to determine when to introduce program changes 

to increase the probability that .students would achieve their^goals. 

By January 4 all teachers had written goals and # objectives, and were 

I 

measuring and graphing student performance. By, February 10 all 
teachers were employing data-utilization rules to determine when to 
make instructional -changes (cf. Mirkin et aK; 1981). 

^sample Goal and Objective Form, Graph, and Instructional Change 

Form, for Michael appear in* Figures 1, 2, and 3, respectively. The 

* 

Goal and Objective Form states that, in the 19 weeks remaining in the 
school year, Michael will improve his reading in Level 2 cf the SRA 
passages s6 that he reads 85 words correct per minute with no more 
than 8 errors (see Figure 1)- This annual goal represents 
approximately J. 5 N times improvement over Michael's baseline rate of 35 
words o6r minute (see Figure 2). This artnual goal then -was divided 
into 19 weekly objectives, each of Which states that he will improve 
at an average Increase of 2.6 words correct each week per minute kith 
,no increase in errors. . ~ „ 



Insert Figures 1-3 about here 



Michael's graph (see Figure 2) represents his w°ords read per . 

« s 

mirtute each school day in Level i of the SRA passages. The first 
three days of data depict Michaels baseline performance; the diagonal 
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1 ine 4 'connecting the baseline median performance with the -X is the 

dynamic aimliney which represents Michael's objective or . the daily' 

rate at, which he must perform in order to meet the annual goal". 

The vertical Vines on Michael 's, graph signify the introduction pf 

program changes, which are indicated by^h^ letters B, C, and D. Each 

program ^change is detailed on the Instructional Change Form (see 

Figure 3). At least two times each. week, MichaeJ read from a randomly 

selected passtfte from Level 2 for one minute.; th^TumEe?^ words he 

read correctly and the number of errors were scored and .graphed on 

Figure 2. Every 7 to 10 data- points, Michael's teacher inspected the 

graph. If the slope, (calculated T>y" the split-median, solution; White, 

1971) of Michael's performance across the 7* to 10 days was less than 

the slope of ' the dynamic aimline, then the t°eacher introduced & 

program change. As Figure 2 illustrates., Michael 1 s/\ performance 

improved dramatically Over his previous performance with the 

'introduction of the tMrd program change. 

I With the Goal and Objective Form," the <M5naph, r and the 

Instructional Change Fortf, Michael's educational program and progress 

toward goals were evaluated formatively. In response to measurement 

data, 'Michael's program was modified throughout the treatment phase to 

improve the likelihood that Michael would achieve his annual goal. 

N > i \ 

In the contrast treatment,; teachers- measured and evaluated 

student progress towa/d goals uiing conventional special education 
procedures. I 1 

Training . Each of three teacher trainers j(TJs) was assigned to a 
set of schools and to the experimental and contrast teachers within 



that set of schools. TTs provided training to teachers during weekly 

meetings from November through May. During these meetings, TTs taught 

the treatment procedures to the experimental group teachers. They 

taught the contrast group teachers strategies for diagnosing and 

. treating learning and.behavior problems, for structuring and, managing 

their instructional groups, and for using audio-visual .equipment and 

paraprofessionals. TTs spent similar amounts . of time with both 

treatment groups: Experimental group teachers received an average of 

23.52/bours individual meeting time x (SD = 5.95), whereas contrast 

group teachers^met individually with trainers" for an average of 20.60 

hours (SD = 6.22). A Jt'test on the difference between the -hours of 

training received by .the two groups revealed no statistically 

significant* difference. (See Appendix CT for sample experimental and 

contrast -training schedules'.) 

> 

Data collection . TTs were taught and practiced the SIRS during a 

five-hour training session. ^ Their infer-rater reliability was .87 on 

» 

two trailing tapes. " In January % April, and \ate May, a randomly 
selected student for each teacher was observed by a TT during a 
20-minute reading lesson. Immediately following these observations, 
TTs completed the SIRS* '* • 

" In April and June, experimental and. contrast grpup teachers 
completed, the Semi-annual Teather Questionnaire for their students. 
In June, the # experimental group' teachers &}$o completed the End-of- 

y i . ; 

the^ye&r Teacher Questionnaire. x 
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Results 

Instructional Structure 

The degree of instructional structure was assessed during each 

i 

trimester of the^study using the SIRS. These scores were subjected to 

a j&ne between factor (treatment), . one within factor (trials on the 

SIPS) analysis of variance* Averaged across the three assessments, 

there was no statistically significant, difference between^ the 

experimental and contrast groups on the total SIRS scores (mean or 

Structure, Positive Consequences, and Silent Reading). There also was 

no statistically significant .difference on the SIRS trials v/hen 

performance was averaged across the experimental and contrast groups. 
* * 
However, there was a statistically significant treatment X SIRS trials 

<< 

interaction, with the experimental group, increasing their structure 
through^the study ' trimesters (2,31, 2.76, 2.98, respectively) and 
contrast teachers decreasing their structure through the treatment . 
.(2.82, 2.52, 2.34), £(2,69)*= 6.57, £ < .01. 

Teacher Decisions about Student Progress, Goals y and Cur rent 
Performance Levels 

On the Semi-annual Teacher Questionnaire, completed in April and" 
June, teachers described student progress, goals, and level of 
•functioning in reading, and' checked the assessment procedures on which 
they relied to formulate' these evaluations. In both April and June, 
there was a statistically significant relation between treatment 
condition and whether • teafchers judged, that their students had made 
sufficient progress to reach their goals, x 2 (2) = 6.92, £ < .05 for 
£pril, ar\d x 2 (Z) = 12.77,- ; £*< .005 for June. For both time periods, 

i 



the pattern of the, relation was the same. .A greater percentage of 
ttachers in the contrast condition thought their students would meet 
goals; a smaller percentage of those teachers reported that their 
students would not meet goals; and a greater percentage of that group' 
was uncertain. 

WithVespect to whether teachers had adjusted^students* reading 
goals, a statistically significant relation with measurement 
evaluation group was found to exis-t in April, x^U) = 4.35 £ < .05. 
Sixty-five percent of experimental group teacjiers reported that they 
had* changed goals, whereas only 33% o* contrast group teachers said 
they had revised goal statements. However, the relation was no longer; 
present by June; at that time, both groups tended to maintain their 
pupils 1 'goals. 

Teachers also were asked to describe the success of the current 
reading program for each stadent. Regardless of treatment group, 
teachers fended .to evaluate their programs as successful (74% and 79% 
for experimental and contrasts, respectively). Very few* teachers 
described their programs as not successful (6% and * 7% for 
experimental and contrasts, respectively). The remaining teachers 
were uncertain about the success of their programs. 

When teachers were asked in April to "write s a precise statement 
that describes the student's current level of functioning," there was 
no relation between the specifiqity of their statements and 0 the 
measurement/evaluation treatment. In June, however,, a relation was 
found, x 2(2) - 9,84, £ < .01, with greater percentages of experimental , 
students 1 programs described highly and somewhat specifically and with 



10 

greater percentages of contrast students 1 pr<ifgrams described no£ at 
all specifically. 

* Figures 4 and 5 display the percentages of teachers' descriptions 
of student progress in April and June, respectively. At both times, 
the contrast teachers rated their students' progress somewhat less 
favorably; in -June, this pattern was statistically significant, x 2 (5) 
f 17.51," £ < .005. 
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Insert Figures 4 and 5, about here 

Assessment Methods on which Teachers Relied 

, The remaining questions on the Semi-annual "Teacher Questionnaire 
concerned -which-assessment^pnocedures teachers relied on to formulate 
their evaluations. At both data t collection times and in both 
treatment groups, teachers r,epor,ted that they -relied on criterion- 
referenced measurement and on informal measurement; very small 
percentages of teachers reported that they relied on standardized 
testing. In April and June, there were statistically significant 
relations between treatment group and teachers' tendency to use direct 
and frequent "measurement, x 2 (l) = 22.28, £ < .001 and X 2 (X) = 29.42, £ 
< .001, respectively. fto contrast group teachers at either time 
reported that they relied on direct and frequent measurement. 

Table 1 presents percentages of teachers in April and June 
reporting that they relied predominantly on different measurement 
procedures. As one can see, at both times, experimental teachers 
tended to rely predominantly on direct and frequent measurement, with 

* , * » 
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a». even greater percentage of experimental relying on this form of 
measurement by the end of the study. Contrast teachers tended to rely 
predominantly on> one of several types of measures: standardized 
diagnostic tests, teacher-made tests v workbook samples, criterion- 
referenced measures (other than direct and' frequent measurement), and 
number of short-term objectives mastered. Approximately the same 
percentage of teachers in the two groups relied on informal 
observation, . The pattern of ^responses was significantly related to 
* treatment condition in both April and June, x^(6) 31 33.46, £ < .001 
and x^(6) = 48.88, £ < .001, respectively. From April to June the^e 
were two shifts in the types of procedures teachers relied on most. 
By June, experimental gcoup teachers tended to replace Aeacher-mader 
tests with direct and frequent measurement; contrast group teachers 
tended to replace teacher-made tests and number of short-term 
objectives mastered with workbook exercises. 



Insert Table 1 about here 



Descriptions of Continuous Evaluation Procedures 

On* the End-of-the-year Teacher Questionnaire, experimental 
teachers typically described the continuous evaluation procedures as 
""very different" \68%) from their normal student evaluation 
procedures; 25% described the procedures as ''different," and 8% 
described them as "somewhat the satne." When asked Io describe the 
ways these procedures differed from their normal routine, graphing was 
named most often, the accuracy of the method was mentioned second most 

• ' "16 
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often, and both measurement frequency and the way \n which pupils were 
motivated were cited third most often\(see Table 2)* Teachers 
reported that the experimental procedures were either "not very time 
onsuming," or "time consuming" (37* in each category); 26% described 
the procedures as "very time consuming." 



Insert Table 2 about here < 



Teachers also were required to rate the usefulness of the 
continuous evaluation procedures *for several different functions. 
Table 3 is a. display of the percentages of teachers giving each rank 
for each function. As one can see, the greatest percentage of 
teachers ranked the continuous evaluation procedures as "very helpful" 
for deciding when, to change students' programs, and as either 
^helpful" or "very 'helpful" for^developing goals, measuring students 1 
progress toward goals, and communicating to parents and Schools. 



* Insert Table 3 about' here 

When asked whether they would use the procedures next year^, 68% 
said "yes," Si said "no," and 26* said ^"maybe." Further, 34X'of those 
indicating they would use the continuous evaluation procedures next 
year reported that they would employ the procedures with no 
modifications; other teachers cited a varifety of changes in the 
'procedures. Among the modifications named most frequently were (a) 
measuring a different reading behavior, (b)- measuring Jess- frequently, 

^_ : tn 
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and (c) starting to implement the procedures in September. With 
respect to the academic domains for which they would employ the 
procedures, teachers most frequently named reading and math, ' with 
social behavior and written expression also« cited. 

Discussion 

The purpose of this study was to explore whether systematic, on- 
going measurement and evaluation procedures affect the instruction 
that special educators provide <*nd the, educational decisions they 
make, toward this end, teaphers were assigned randomly to either an 
experimental or contrast treatment group. In the' experimental 
treatment (Mirkin' et al., 1981), teachers were to (a) measure and 
graph students 1 oral reading nerformance at least twice weekly, and 
(b) compare at frequent intervals L their students 1 actual progress 
trends against the slopes of students' expected goal ajmlines. When 
students' actual procress trends compared unfavorably wUh, their 
expected aimlines, teachers were to introduce ci new dimension into 
thetr students' programs in ah attempt to stimulate greater student 
progress. Contrast teachers employed conventional special education 
measurement and evaluation^ procedures. During the study, the 
structure of the instruction provided by these teachers was measured 
three* times. The teachers* educational decisions and their assessment 
procedures were surveyed at the middle and end of the study. 
Additionally, t at the termination of the treatment, teachers' 
descriptions and ratings of the experimental procedures were assessed. 

Results indicated that using systematic measurement and 
evaluation procedures may increase the degree of structure in the 

: '. 18. : 
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instruction teachers provide. Structure scores across the three 
trimesters of the study increased for experimental group teachers, but 
decreased for contrast group teachers. Research suggests . that 
increased structure contributes to handicapped children's achievement 
(Retth et al., 1981; Stevens & Rosenshine, 1981), and may explain at 
least partially why, by the end of the study, students of the 
experimental teachers demonstrated better reading achievement than 
students of the contrast teachers (Fuchs, Deno, & Mirk in, 198?). 

Findings of this study also suggest that teachers' decisions 
about student progress, goals, and current performance levels were 
related to how teachers measured and, evaluated student progress.' In 
June, when asked to rate student progress on a 5-point Likert type 
scale", experimental . group teachers tended to give more 1 favorable 
ratings. Nevertheless, both in April and in June, contrast group 
teachers were both, more optimistic and more uncertain than 
experimental group teachers about whether btudent progress was 
adequate" to attain goals. This uncertainty may have been a function 
of a data base that was inadequate relative to that of the 
experimental group teachers. The contrast teachers' optimism is 
explained "less easily, because their students actually achieved poorly 
compared to students of the experimental group teachers. One might 
speculate that their optimism concerning student progress toward goals 
was related either to their having formulated less amoitious goals or 
to their misjudging - studeht progress. This second explanation is 
supported in two* ways: first, by ^research (Fuchs, Fuchs, & Warren, 
1982) that demonstrates a tendency for teachers to, overestimate 

19 
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student performance on goals when assessments are based on 
unsystematic procedures; second, by results of this study suggesting 
that the experimental teachers tended to describe their students' 
current performance - ~1eVeTs~ with - much greater specificity than the ' 

0 

contrast group teachers* » « 

Additionally, contrast group teachers apparently .overestimated 
the success of their 'programs. In Both April 'and June, nearly 
equivalent and very high percentages of teachers in both groups rated 
their programs as successful. Given the greater achievement of the. 
experimental group students, these nearly equivalent percentages of 
"teachers' ratings of program success again suggest inappropriate^ 
optimism by the contrast group teachers. 

Perhaps related to the contrast group teachers' comparatively 
optimistic view on student progress toward goals and on their programs 
was their tendency . to maintain established goal statements. 
Experimental group teachers more frequently reported that they revised 
their students' goals during the firsi two-thirds, of the study. Given 
the difficulty of establishing initially appropriate goals for all 
students (.Fuchs & Deuo, 1982), one might anticipate a need to revise 
many goal statements. Yet, contrast group teachers failed to do so. 
It is possible that, based on unsysteu-atic and lean data, they 
misjudged student performance as adequate and were content to maintain 
goal statements. Experimental teachers, on the other hand/ may have 
demonstrated greater responsiveness to student behavior because of 
their comparatively rich data base; they may have had more and better 
information with which to recognize when goals failed to suit 

\ 
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students 1 current performance levels and when such goals needed tp be 
revised. 

* Results concerning which* assessment procedures teachers relied on 
to' formulate -their decisions- ^houtlstudent .^progress,. . j^al?>_ and 
current performance levels corroborated the assumption that teachers 
in the two treatment groups actually employed different data bases. 
Throughout the study/ large percentages of experimental group 
teachers , but no contrast group teachers, • relied on direct and 
frequent ' measurement. Additionally, teachers described the 
experimental measurement and evaluation procedures as very different 
or different from typical special education practice. Furthermore, 
from April to June, experimental group teachers shifted their reliance 
from less ' frequent and systematic procedures to more frequent and 
systematic procedures (from teacher-made tests to direct and frequent 
measurement), in ^contradistinction, contrast group teachers tended to 
shift their reliance to less systematic procedures (from teacher-made 
tests and nymber of short-term objectives mastered to workbook 
x samples). , • 

* Consequently, findings of this study t suggest that, concurrent 
with a better data base and with improved' pupil gains, teacher 
decision ^makin^K^nd instruction improved when continuous ^evaluation 
procedures were usetiv Compared to contrast teachers who* used typical 
special education measurement and evaluation methods, experimental 
teachers, who employed and, t^^Urge extent, relied predominantly on 
technically adequate, repeated, ^rriculum-based measurement and 
evaluation procedures, not only achieved fetter student academic gains 
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(Fuchs, Deno, & Mirkip, . 1982), but, also (a) demonstrated greater 
increases in the structure of the instruction they praviJed, (b) were 
more realistic about and responsive to student progress toward goals, 
(c) judged toore accurately* the success of .their students' programs,- 

(s more 



and (,d) described their students' current performance 
specifically. 

In this study, teachers tended to indicate that the continuous 
evaluation" o procedures were either "not very time consuming" or "time^ 
consuming, " Feasibility problems . with implementing frequent 
measurement procedures have been demonstrated previous 1 ' (King, 
Wes3on, & Deno, 1982; Tindal, Fuchs, Christenson, Mirkjn, & 'Deno, 
1981) • Nevefthelpss, teachers rated the pr6cedures as useful or very 
useful for an 'array of parposes, and most teachers reported that they' 
would continue to use the procedures next year* Therefore, although 

4 I 

t 

the procedures may be somewhat time consuming, teachers appear to 
recognize the procedures 1 potential usefulness and benefits, which 
were demonstrated empirically in this study... 
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. Table*! 

Percentages of Teachers Relying on 5 Different 
Measurement Procedures 
4> 



Measurement Proce'dure 



April ■ 
Experiment^ Contrast 



Experimental " Contrast 



Standardized diagnostic 


■ •• • 


6 , 


• 0 : 


6 


Cri tef i bn-ref erencfed 


0 / 

/ 


10 


' 5 


' 1 10 


uirect ana Trequent 


Aft* 


n 

V 


60 


d 

V 












Teacher-made ^ 


21 


38 


• " 8'* 


29 


Workbook samples 


0 


6 


' 0 


- 26 


Informal observation' 


24 


23 


25 


/ . 21 


V 

Number of short-term 


8 


V7 

• 


3- 


$ 8 


objectives" ma.s^Ked 
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* ./: Table 2 . ^ 

Frequency, with which Teachers C1 ted -Different? Aspects o^Contlnuo us 
Evaluation Procedures as Most Distinctive 



0 


* N • 
Cited First 
* 


N • 

Cited Second 


N 

Cited Third 


Total 
N . 


* ' ' 

urdpn iny , • 


20 ' 


l> 

10 


7 


37 


Accuracy , t fc 


21 


7 


2' . 


30 


Frequency 


17 
* * • 


4 


0, 


' ,21 
» 


Motivating to Child 


0 


14, 


• 7 


21 


* 

Permanent Record 


. . 3 

* 


0 


■ 4 

G 


"7 


Type of Behavior Measured 


4 ; " 

* 


0 

\ 
i 


* 0! ^ 


4 


Curriculum-based 

v ■ ■ ~ — = * — 


0 


2 * " 


' 0 


\ 



,4> 



• / 



- 26 



i 
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Table 3 

Percentages of Teachers Rating the Continuous Evaluation Procedures 
- °as, "Not Sure" through "Very Helpful" tor Different Purposes 



Purpose 



a. 
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Q. 

O 
f — 
" V 

> 
qj 
o 





VI 




trt 


in 


Q) 
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CD 


S- 
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u to 


og 


; E o 

E SZ. 


mm 
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re 


pn 

LUUI 


QJ sz 
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03 


O 4J 


o o 




o to 




a. 


o to 



X 



■■ - » v " — 

Not Sure . . * 
Not Helpful 


K * 










2 




• 6 

31.. . 


Somewhat Helpful 




36 


18 


.5 


23 • 


28 


17 


12 


Helpful 




* 

32 


37 


50 " • 


22 


40 


37 


28 


Very Hel pful 




" 31 


45 * 


45 


55 


30' 


46 


23 



ERIC * 



27 a 



' \ 23 

GOAL If fey v 19- weeks , when provided with 

" | school weeks until year> end ** 

- stories from grade level 2 - SRA passages' , Michael 

(level §, series) 



* . 

will, read aloud at the rate of .85 

(wpm correct) 



(student's- name) 
, with no more 



than 8 

— m 



errors. . 



OBJECTIVE Each- successive week, when presented with a random selection 

-from Grade level 2 - SRA passages , the student will read 
(same level I and "series as above) 

« * * 

aloud at an average increase of 2.6 wpm and no increase in 

* .H7T . , 

errors. « 



Figure 1. Michael's Goal and Objective Form 
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• CORRECT 
o ERRORS 



100 - 



50 - 




MONTHS 



r 29' 



Fiaure 2 Number of Correct Words (•) and Errors (o) Per Minute Read by Michael from Pages 
Figure 2, . sS, Level 2 AcrosslTlwie, Under Baseline (a) and Three Instructional Strategies 

(B, C, and D) . 
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Instructional Change Form 



Instructional Procedures 


Arrangement 


Time 


Materials 


1 , * 

Motivational Strategies 


Oral RcaOing rractict 
Comprehension exercises 


GrouD (1:5) 


45 minutes 


Double Action Short Story, 


Genera* in'g* own stories 




Part 2 

Story Writing & class 
discussion 




Language Experience 
Approach. 

i 


Individual 
with para- 
professional 


same 

** 


Student's own stoye^s 
File cards 
Story Folder 


same 


Language Experience, 

Reading Comprehension 
Activities, 


Individual' 
with para- 
.professional 

Individual, 
with teacher 

4 — • * c 


20. minutes 
20 minutes 


See above 

McCall-Crabbs, Book E 
SRA kit 


same 

individual arrangement 
with teacher 
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Figure 3. Michael's Instructional Change Form, 

31 
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A — A Contrast- 




— I — 
Poor 



— h- 
Fair 



Good 



Very Rood Excellent 



- ' Teacher Descriptions 

t 

Figure 4. Percentages of Teacher Description's of Student Progress 

- in April . 
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40 
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figure 5. Percentages of teacher descriptions of student' 
progress in June. 6 
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Append fx* ' 



Structure of Instruction Rating Scale (SIRS) 
School: — Student:__ 



Date:_ ™ cher: : 

Observer Number of Stud6ntS 1 " Gr ° UP: - 



Number of observations prior to rating: — . l. 

.1 l. * , Time observation ends: 

Time. observation begins: . nme 9Use ' 

Time allocated to reading Instruction per day: ____ 

Currlrulum used for Instruction: Publisher 

Series ; Level . 

Instructions s 

Circle the number that accurately reflects, your rating for each 

to the left-hand column. 

* 

1. Instructional Grouping 

2. Teacher-directed Learning 

3. Active Academic Responding 

4. . Demonstration/Prompting 

5. Controlled Practice 

6. Frequency of Correct Answers 

7. Independent Practice 
8'. ■ Corrections 
v 9. Positive Consequences 

10. Pacing 

t 

11. "Oral practice on Outcome 

Behavior 

12. Silent Practise on Outcome 

Behavior 



1 


2 


3 


4 


5 


1 


2 


,3 


4 


5 


.1 


2 


3 


4 


5 


1 


2. 


3 


4 


1 5 


1 


2 


3 


4 


5 


1 


2 


3 


4 


5 


1 


2 


3 


4 


5 


1 


2 


3 


4 


,5 


1 


2 


3 


4 


5 


1 


2 


3 


4 


5, 


1 


1 


'3 


4 


s 


1 


2 


3 


4 


5 
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SIRS 

Operational Definitions Codebook 



1 . Instructional Grouping 

5 - 90X or more <yf the instruction this student receives from the 
# teacher is 6h an individual basis, , 

1 » 1 0X or less of th? instruction this student receives from the 
teacher is on an individual basns. 

2. Teache r- Direc ted Learning 

5 - Student's instruction is extremely organized, businesslike, 
and teacher is firm in direction and control of activities. 
For' example, student is -presented with questions, student 
has material to cover, etc, 

1 - Student's instruction is casually organized and very spon- 
taneous. Teacher is- not committed to having the student work' 
on a particular set of material . Instructional materials do 
not determine what activities student engages in and the les- 
sons change according to problems or mood of this student. 

• * 

3. Active Academic Responding 

5 - The student is actively practicing the academic skills to be 
learned more than 75X of the time" observed, Speci fically,. the 
, s-tudent is engaged in oral or written, responding to teacher . 
questions or written material, e.g., reading ajoud, answering 
questions, writing, or computing. Student rarely is involved 
in non-academic conversations with teacher or other students 
Attending to the lesson without responding", such as sitting, 
looking, listening, and/or fol lowing along in a book does not 
apply. The student Inust make an active, written or oral 
. response. , « 

1 - The student is actively practicing the skills. to be learned 
less than 10% of -the time observed,. Instructional lessons 
may be interrupted or shortened to include "process" and other 
non-academic activities, e.g., clarifying feelings, opinions, 
■ and working on arts and crafts. , \ 

4. Demonstration and Prompting \ 

5 - Appropriate steps, of the desired beh'avior to be performed are 
demonstrated for the student. Student is givetv an opportunity 
to practice the step(s) as teacher provides prompts for correct 
behavior that approximates or achives desired response. 

1 - Teacher attempts to teach the student a behavior without using 
demonstration and prompting techniques. \ 
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5. Controlled Practice * / ' * 

; ' ! * 

5 - Student's practice of material Is actively controlled by 
teacher Who frequently asks questions to clarify that the 
student understands what has just been demonstrated. Ques- 
tions are convergent ( single factual answer ) and the -stu- 
dent's answers consistently follow the questions and are 
given teacher feedback. 

* « 

1 - Student Is rarely questioned* by teacher following demonstra- 
tion of new materials. Questions are more divergent (open- 
ended, several Interpretations) than convergent (single .factual 
answer),. Student's response 1s not consistently followed by • 
teacher feedback. The type o.f questions are such that several 
answers are acceptable, I.e., questions are abstract or am- * 
biguous.- 



Examples: 

If during an oral reading session: 

a) the teacher frequently attempts to clarify >the material with 
convergent questions ("what color hat was John wearing? ), a 
5 would be recorded. 

b) ' the teacher asks few questions, most of which are divergent 

("What do you think this, means?") , a 1 would be recorded. 

c) the teacher asks few convergent questions or -^nv. divergent 
questions, the appropriate rating would be a'3. . 

6i 4 Frequency of Correct Answers 

5 - Academic lessons are conducted in such a, way that the difficulty 
of the material allows the student to achieve mean accuracy 
of 80% or higher. . 

1 - Academic .haterial '1s difficult for student, component steps 
are large or unsequejiced, and mean accuracy for student Is 
less, than 55%. , 

(Note: -if the student has no opportunity for oral or written response 
during the observational period, item 6 would be rated N/A - 
not applicable, while items 3 and 5 would most likely be 
rated 1 ) . 

7. Independent Practice 

5— When engaged in independent seatwork, the student frequently 1s 
monitored bythe teacher who assists, clarifies, and praises 
the student for academic engaged tasks. . 

(Note: independent seatwork |ts defined here as a student working on an 
assigned task for at least 5 minutes. Uf no such 5-mmute 
block of time is observed, Item 7 1s rated N/Aj.) 

l_oja * 
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S^RS 

1 - When student 1s engaged tn academic seat-work activities, little 
attention t 1s given by teacher who directs seat-work activities 
from a distance or engages 1n work separate from the assigned 
seat work. Teacher is generally not helpful or supportive to 
student during Independent practice time;* 

8. Corrections * • 

V 

5 - The students errors are consistently corrected by the teacher. 
Jrfhen the student- either "does not respond, responds Incorrectly, 
or does not respond 1n unison 1f the activity is group directed 
and requires such responding, the teacher will systematically 
* attempt to correct the student by asking a simpler question, re- 
focusing student's attention to elicit correct response from the 
student or provide general rules by which to determine the 
correct answer* 7 90% or more of the time. 

* 1 - Student's errors are rarely and inconsistently corrected by the 
teacher. The student responses are not systematically corrected. 
Student's errors are corrected 50% or less of the time. 



For example: In oral reading this includes teacher correction of skips 
and mispronunciations, or help in sounding out hesitations. 

9. Positive Consequences a 

5 - Positive event* (tokens, "points, activities, etc.) are* given to 
the student when performing the desired behfvior. When learning 
a new skill the student receives positive consequence for 
approximations of the. desiredbehavior . Consequences are con- 
sistently received during academic training time.^ Praisejand 
compliments, e.g., "gopa\.w6r king, nice job," are Wt included 
in this definition. ' 

: -> Student rarely receives positive consequences for academic work. 
When student receives consequences they usually, are for social 
behavior, rather than- for behaviors occurring under systematic „ 
'academic training. , 

10. Pacing * , 

5 - The pace of the lesson is rapid, providing many opportunities 
for response by the student. As a result, attention is high 
and off-task behavior is low. * 

< * ' 

1 - The pace of the lesson is slow and the student's rate of 
■ responding is low. Lesson format frequently varies, is not 
highly structured, and student attention may be" low. 
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SIRS 

11. Oral Practice on Outcome Behavior * 

5 - Student reads aloud from context nearly all the time (85-1 00%. 
"or 12-15 mi n. of a 15 m1n $ . observation). 

1 - Student does not read aloud during the observation (OX of thejo 

Jti'nieh- •• ♦ 

(Note: Reading aloud for measurement purposes should not be considered 
> '-when rating tM$ variable. Reading in context is defined as 
reading phrases,, sentences, paragraphs, or story selections.) 

Examples: . 

If.the student is reading Isolated words nearly the entire time, 
the appropriate rating 1s .a 3. \ 

If the student 1s reading aloud from a text about half the time, 
a 3 would be* recorded. m p 

i - * 

12. Silent Practice ori Outcome Behavior - 

5 - Student reads silently from context nearly all the time (85-100X 
or 12-15 min. of a T5 min. observation). 

1 - Student does not read silently during the observation (OX 'of 
the time) . 

(Ndte: Reading in context is defined as the same as #11 f The examples 
of #11 are the sane for #12, with silent reading.) 

•V' • 



•Appendix B - 
• ■' Semi -Annual Teacher Questionnaire 



Student's Name* .' * ' > . • 0ate - 



* 



Teacher r s .Name 



1) Is the student making sufficient progress in reading so that he/she will 
-meet the annual reading goal? J—^| |-— j J 

. . Yes' No 'Uncertain 

2) Have you changed your reading goa.l within the last two months? 

■ " • □ P ,P : ■ ■' 

, ' . ■ Yes No Uncertain 

' \ * 

3) Is your current reading program successful? . * : 

>; ' ' Yes , No Uncertain 

41 For reading, please write a. precise statement that describes the student's 
current level of functioniK or indicate that>u are uncertain about the • 
student's current level of functioning. - . 

fnrr ent Level of Functioning * f • • • 

• • : ~l- ■ 



Uncertain 

5) Please characterize this student's progress so far this yearly one of the 
following descriptions: 

o- n u □ JU 



Poof Fair Good ; Very Exce 

"Good 



lent 



Check (/) the tyoe(s) of information you used to answer question five. £ll|le 
the one type of information, you rely on the most to provide an indication or . 
student progress; ' , . 

"l Standardized achievement tests JiJU Informal observation of stu- 
— t I Sardized diagnostic measures ' ^nt^erformanc 

— Si XK«3^^5»"« ^~ teacher regarding classroom. • 

—-6. Direct and frequent measurement perfor man cfr obiect1ves 

7. Teacher-made tests/oral quizzes 14. Number of .short-term objectives 

n ,.,„.. t,k„„i,c mastered 



"8. Scoring workbooks J«wr w 

"9. Scoring worksheets- *> L 15< other — — 13 s 

"10. Amount of work completed 



Make sure you have circled the type of information you rely ort .the most to 
pnovide an indication of student progress. 



END-OF-YEAR SPECIAL TEACHER QUESTIONNAIRE* 
Student's Name ■' Date 



1) How different were the continuous evaluation procedures you Implemented 
from your normal procedures for evaluating student progress? 

* i 

' ' □ ' □ □ □ • 

Very Somewhat Very much 

Different Different the same the same 

2) In what ways io the continuous evaluation procedures you implemented 
.differ from your normal procedures for evaluating student progress,? 



a. 



c. 



/ 



How useful were the continuous evaluation- procedures in: 
-3) Developing the student's IEP? 

□ - d □ □ □ ' . 

Not -sure Not Somewhat 'Helpful Very • 

Helpful' Helpful Helpful j 

41 Establishing goals? 

Not sure Nat ' Somewhat Helpful Very 

HelpfuT Helpful Helpful 

5) Measuring progress toward goals? f 

•P ' • □ r □ □ □ 

Not sure Not Somewhat Helpful- Very 

Helpful Helpful Helpful , 

6) Deciding when to change the student's program? ' 

. d □ o □ / 

Not surle Not Somewhat Helpful Very . 

| Helpful Helpful Helpful 1 



B-3 

?) Communicating evaluation information to' school .personnel ? 

□ ■ ta □ . '.□ □ 

Not sure Not Somewhat- Helpful Very 
Helpful helpful 1 Helpful 

8) Communicating evaluation information to parents? 

< □ * □. ' □ '" . □ 

Not sure Not So|newhat Helpful Very - . 

Helpful helpful . Helpful m 

9} Communicating evaluation information to students? • . 

D« □ P P 

Not sure Not Somewhat Helpful' Very 
Helpful helpful Helpful 

1Q) 'mong the uses cited above, please rank'order'3 purposes for which 
the procedures were most useful: 



most. useful for 



second most useful. for_ 
thiVcf most useful for 



11) How time consuming were the procedures? 

Very time Time- Not Very Not at all 
"* Consuming- Consuming Time Time 

p *' Consuming Consuming 

12) If you had a choice, would you use the" procedures next year in some way? 

□ □• □ * 

Yes No Maybe 

t 

If yes* please describe briefly what, if any, modifications you will 
make in the procedures? 



If yes, in what academic areas will you use these procedures? 



Appendix C 
Goj.1 and Objective Form 



GOAL In ' , when presented with' stories from 

'(# school weeks until. year's end) 

wil 1 read aloud at the 



(Level f, series) 
rate of 



(wpm correct) 



(student's name) 

with no more than 



errors 



OBJECTIVE Each successive week, when presented wittua random selection from 

^ , student will read aloud at 

(same level 1 and series as above) 

» * * 

an average increase .of wpm and no increase in errors • 

UT 

MEASUREMENT SYSTEM Frequency: at least ZX/week; stimulus format: oral 

reading passages; Test administration procedure: same as manual; Scoring 

procedure: same as manual; charting conventions: same as manual. 



,erJc 
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Appendix D 
Sample Experimental Training Schedule 



Teacher Trainer 
Teacher 

School 



Date 



pplnw nlease' (a) describe the nature of the training you. have provided 
this Ser'duHng (6) indicate ^""'^"fftf 

under which the training was provided (individual. J:* ™£V*ih training, 
(c) estimate the number of hours you spent with the teacher on acn 

area. 



DESCRIPTION OF TRAINING 
data-based monitoring procedures 



ARRANGEMENT 
individual 



NUMBER OF HOURS 
15 hrs. 
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D-2 . . 

Sample Contrast Training Schedule 

Teacher Trainer ' Date _ 

Teacher ; ' 

^^J-5cT»ool ____ 

Below, please: (a) describe the nature of tha training you have provided 
this teacher during the study, (b) indicate the administrative arrangement 
- under which the trailing was provided (individual, 1-3 ratio, etc.), and • - 
,(c) estimate'the number of hours you spent with the teacher on each training 
areaf *?- 



DESCRIPTION. OF TRAINING - 


ARRANGEMENT 


NUMBER OF HQURS 


Jsing the language 
experience approach 


individual 


10 hrs. 


Individualizing 


"individual 


5 hrs' 


- 

Jsing audio-visual equipment 
for teaching and motivating 


individual 


2 hrs. 


diagnostic assessment tools 


individual 

• 


3 hrs. 

•« 




• 

* 






\ 
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